Unfinished Test Assertion

void testSomething() {
   // Outline:
      // create a flight in ... state
      // call the ... method
      // verify flight is in ... state
   fail("Unfinished Test!");
}

As we start defining the tests for a particular piece of code, it is useful to "rough in" the tests by defining Test Methods (page X) on the appropriate Testcase Class (page X) as we think of the test conditions. We do, however, want to ensure that we don't accidently forget to fill in the bodies of these tests if we get distracted. We want the tests to fail until we finish coding them.

Including an Unfinished Test Assertion is a good way to make sure we don't forget.

How It Works

We put a single call to fail in each Test Method as we define it. The fail method is a Single Outcome Assertion (see Assertion Method on page X) that always fails. Include the Assertion Message (page X) "Unfinished Test" as an easy way to remind ourself of why the test is failing when we do run the tests.

When To Use It

We should not deliberately write any tests that might accidently pass. A failing test makes a good reminder that we have work to do. We can remind ourselves of this work by putting an Unfinished Test Assertion at the end of every test we write and only removing it once we are satisfied that the test is coded properly. There is no real cost to doing this and a lot of benefit. It is just a matter of getting into the habit. Some IDEs even help us out by letting us put the Unfinished Test Assertion into the template for a "testmethod".

If we need to check in the tests before all the code is working, we shouldn't remove the tests or the Unfinished Test Assertions just to get a green bar as this could result in Lost Tests (see Production Bugs on page X). Instead, we can add an [Ignore] attribute to the test if our member of the xUnit family supports it, rename the test method if xUnit uses name-based Test Discovery (page X) or exclude the entire Testcase Class from the AllTests Suite (see Named Test Suite on page X) if we are using Test Enumeration (page X) at the suite level.

Implementation Notes

Most members of the xUnit family have fail already defined. If the member that we are using doesn't include it, we should avoid the temptation to sprinkle assertTrue(false) throughout our code. This is obtuse and easy to get wrong because it is counter-intuitive. We should take the few moments to write it ourself as a Custom Assertion (page X) and write the Custom Assertion Test (see Custom Assertion) for it first.

Some IDEs includes the ability to customize code generation templates. Some even include a template for a Test Method that includes an Unfinished Test Assertion.

Motivating Example

Consider the following Testcase Class that we are roughing in:

   public void testPull_emptyStack() {
     
   }
  
   public void testPull_oneItemOnStack () {
     
   }
  
   public void testPull_twoItemsOnStack () {
      //Todo: Write this test
   }
  
   public void testPull_oneItemsRemainingOnStack () {
      //Todo: Write this test
   }
Example EmptyTestMethod embedded from java/com/xunitpatterns/misc/EmptyTestMethods.java

Including the // todo: ... comments may remind us that the test still needs work if our IDE supports that feature but it won't remind us when we run the tests. Running this Testcase Class will result in a green bar even though we may not have implemented our stack at all!

Refactoring Notes

All we need to do is add the following line to each test as we rough it in:

fail("Unfinished Test!");

The exclamation mark is optional. It might be even better to create a Custom Assertion such as:

unfinishedTest();

This would allow us to find all the calls easily using the "search for references" feature of our IDE.

Example: Unfinished Test Assertion

Here are the tests with an Unfinished Test Assertion added to each one:

   public void testPull_emptyStack() {
      unfinishedTest();
   }
  
   public void testPull_oneItemOnStack () {
      unfinishedTest();
   }
  
   public void testPull_twoItemsOnStack() {
      unfinishedTest();
   }
  
   public void testPull_oneItemsRemainingOnStack () {
      unfinishedTest();
   }
  
   private void unfinishedTest() {
      fail("Test not implemented!");
   }
Example TestNotImplementedAssertion embedded from java/com/xunitpatterns/misc/NotImplementedAssertion.java

Now we have a Testcase Class that can be guaranteed to fail until we finish writing the code and the tests and the failing tests act as a "todo list" for writing the tests.

Example: Unfinished Test Method generation from template

Eclipse (version 3.0) is an example of an IDE that includes the ability to customize templates. It includes a template called testmethod that inserts the following code into our Testcase Class:

public void testname() throws Exception {
   fail("ClassName::testname not implemented");
}

"ClassName" is filled in automatically and as the test name is modified in the signature, the test name in the fail statement is adjusted automatically. All we have to do is type "testmethod" and then press CTRL-SPACEBAR.

Page generated at Wed Feb 09 16:39:38 +1100 2011

Copyright © 2003-2008 Gerard Meszaros all rights reserved